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Neuronal responses are conspicuously variable. We focus 
on one particular aspect of that variability: the precision of 
action potential timing. We show that for common models of 
noisy spike generation, elementary considerations imply that 
such variability is a function of the input, and can be made 
arbitrarily large or small by a suitable choice of inputs. Our 
considerations are expected to extend to virtually any mech- 
anism of spike generation, and we illustrate them with data 
from the visual pathway. Thus, a simplification usually made 
in the application of information theory to neural processing 
is violated: noise is not independent of the message. How- 
ever, we also show the existence of error-correcting topolo- 
gies, which can achieve better timing reliability than their 
components. 



I. INTRODUCTION 

Brains represent signals by sequences of identical ac- 
tion potentials or spikes Q| . Upon presentation of a stim- 
ulus, a given neuron will produce a certain pattern of 
spikes; upon repeating the stimulus, the pattern may re- 
peat spike for spike in a highly reliable fashion, or may 
be similar only " rate- wise" , or some spikes may be re- 
peatable and others not. If individual spikes can be rec- 
ognized and tracked across trials, then their timing accu- 
racy can be ascertained unambiguously; this is not always 
the case. The existence of repeatable spike patterns and 
the reliability of their timing changes not only from neu- 
ron to neuron, but even for the same neuron under vary- 
ing circumstances. Bryant and Segundo j|] first noticed 
(in the mollusk Aplysia) that spike timing accuracy de- 
pends on the particulars of the input driving the neuron. 
This intriguing property has received renewed attention, 
notably among those searching for experimental evidence 
supporting different theories of neural coding . In a 
study of the response of pyramidal neurons to injected 
current j^] the temporal pattern of firing was found to 
be unreliable when the injected current was constant, 
but highly reliable when the input current was "noisy" 
and contained high frequency components. This study 
showed explicitly the difference between the irregularity 
of the spike pattern (which was irregular in both cases) 
as opposed to the reliability or accuracy of spike tim- 
ing, and also highlighted the connection that "natural" 



stimuli are noisy and contain sharp transitions. Similar 
results have been obtained from in vivo recordings of the 
HI neuron in the visual system of the fly constant 
stimuli produced unreliable or irreproducible spike firing 
patterns, but noisy input signals deemed to be more "nat- 
ural" and representative of a fly in flight yielded much 
more reproducible spike patterns, in terms of both tim- 
ing and counting precision. Although some aspects of the 
last study have been challenged recently in 0, the phe- 
nomenon of different reliabilities for varying inputs was 
reaffirmed: see e.g. Fig. lin||. Thus one concludes that 
similar observations of the timing precision of spikes can 
be made in very different types of neurons under vastly 
different conditions, and that this must be a very univer- 
sal, almost basic characteristic of neuronal function. We 
shall now show how these experimental observations fol- 
low from rather general and elementary considerations, 
and discuss the possible implications for brain function 
and architecture. 



II. STOCHASTIC SPIKING MODELS 

The essence of this phenomenon can be demonstrated 
easily in toy models. A simple example is the leaky 
integrate-and-fire (I-F) model || , which assumes the neu- 
ron is a (leaky) capacitor driven by a current which sim- 
ulates the actual synaptic inputs. We add membrane 
fluctuations representing several internal sources of noise 
(cluttering of ion channels, synaptic noise, etc.) to obtain 
a system described by the following Langevin equa- 
tion: 



CV = -gV + I(t)+Z(t) 



(1) 



where C is the cell membrane capacitance, V the mem- 
brane potential, gV the leakage term (g is a conduc- 
tance), I(t) is the input current, and £(i) is Gaus- 
sian noise, (£{t)) = with autocorrelation (£(£)£(£')) = 
a5(t — t'); when the potential reaches the threshold Vq an 
action potential is generated and the system returns to 
the equilibrium potential V e , here set arbitrarily to zero. 

Fig.l shows the results of a numerical simulation of 
Eq. (0) in response to two different signals fllCj , in both 
cases in the presence of identical "internal" noise. Follow- 
ing refs. the left panels show a time independent 
input and the neuronal responses to repeated stimula- 
tion, while the right panels ensue from a "noisy" input 
and its responses. When I(t) contains high frequency 
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components (right side panels) spikes are clustered more 
tightly to the upward strokes of the input and overall 
there is less trial-to-trial variability than in the time- 
independent case. The phenomenology described in Fig. 
1 captures the main feature that renewed interest in this 
subject; i.e., that apparently more "noisy" (in the sense 
of "severely fluctuating") inputs can generate less vari- 
able responses. 



firing time is essentially the time during which the mem- 
brane potential is at a distance from the threshold smaller 
than the value of the voltage noise, and so is scaled by 
the derivative at this particular moment. Thus, the faster 
the voltage approaches the threshold ( in the vecinity of the 
threshold), the more reliable the timing of the spike will 
be. The statement captures much of the phenomenology 
in , and in our own data, as we will show. 
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FIG. 1. Rapidly fluctuating (panels on the right) inputs pro- 
duce more reliable output spikes than a time independent stimuli 
(panels on the left) . Panels in the top correspond to input stimulus 
- I(t) in Eq. 1; middle ones to the 500 responses to the stimulus 
where the occurrence of each spike is denoted with a dot; and bot- 
tom panels depict the peri-stimulus time histogram describing the 
rate - p(t) - at which spikes are generated in response to the stim- 
ulus I(t). 

By concentrating on these two extreme stimuli, these 
studies have somewhat obscured an issue which is cen- 
tral to the variability phenomenon: the relationship of 
the output to the timescale or timescales over which the 
input itself varies. The constant stimulus can be thought 
to have no timescale, or at best, just a timescale since on- 
set; while in principle the noise process has indefinitely 
fast timescales. Thus the two extreme cases under dis- 
cussion have timescales outside the range of interaction 
with physiological processes. 

A simple dimensional argument shows that timing vari- 
ability is dependent on the characteristic timescale of the 
input. As a consequence of noise, the neuron's membrane 
potential will have fluctuations of a characteristic size (or 
root mean square) AV. In order to translate this AV into 
a timing noise At, we have to scale the potential noise 
by something having units of At/ AV; the most natural 
such quantity at hand is the inverse of the potential's 
time derivative. 

While the distribution of the derivative of the poten- 
tial is a reasonable measure of the degree of reliability of 
a particular input, one can see from a geometrical argu- 
ment that the derivative of the potential at the particular 
time in which the potential crosses the threshold is a bet- 
ter estimator of stimulus reliability. The uncertainty in 
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FIG. 2. Computed mean spike temporal jitter (and SEM) as a 
function of the estimated membrane potential's dV/dt. Estimates 
are plotted for two noise variances: a = 0.01 (o) and a = 0.02 
(*). The expected inverse-law fit (with exponent ~ 0.35) is de- 
picted with dotted lines. The examples in the insets illustrate two 
density distributions of spiking times resulting from relatively slow 
(inset A) or relatively fast (inset B) dV/dt threshold crossings. For 
each case, below each histograms, a few typical trajectories are also 
plotted showing the membrane potential preceding the threshold 
crossing. 

We proceed now to apply this method of estimation 
to the situation shown in Fig. 1. A rapidly fluctuat- 
ing signal was constructed ]I(J and the times at which 
spikes occurred were determined first for the determinis- 
tic condition (i.e., a = 0), and subsequently for several 
hundreds stochastic realizations (i.e., with identical J(t) 
but with different stochastic realizations of the noise term 
in Eq. 1). At the same time the voltage derivative pre- 
ceding each spike was computed over the prior fifty time 
steps. The fluctuating character of I(t) (as illustrated 
in the top right panel of Fig. 1) offers the opportunity 
to explore a wide range of dV/dt at threshold crossings. 
To estimate the precision of spike timing we define an 
index of "temporal jitter" as follow: it is the absolute 
difference between the time at which a spike is generated 
in the noise-free simulation and that of the correspond- 
ing nearest spike for the stochastic realization, averaged 
over realizations, and over all spikes which occurr within 
a dV/dt interval of 0.1. This quantity is plotted in Fig. 
2. The data points in Fig. 2 are scattered around the 
expected (dotted lines) inverse relationship between the 
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temporal jitter of the spike and the speed at which the 
voltage crossed the threshold. The trajectories plotted 
in both insets help to visualize the geometrical argument 
already discussed: inputs that rise more rapidly to the 
membrane potential threshold produce less variable spike 
timing. 

The same basic phenomenon will affect any model of 
spike generation, in which the noise in the voltage-like 
variable is translated into jitter of its threshold crossings. 
To provide for a specific example, we choose a widely used 
model of excitable media with continuous dynamics, the 
FitzHugh-Nagumo model (FHN) with additive stochastic 
forcing fTl| ] , described by the following Langevin system: 

V = V — V 3 /3 ~W + I(t) + (2) 
W = 4>{V + a - bW) (3) 

The variable V is the fast voltage-like variable, W is 
the slow recovery variable; a, b and <f> are constant pa- 
rameters, and £(t) is a zero-mean Gaussian white noise of 
intensity D. According to these equations, after a spike 
the recovery produces an absolute refractory period dur- 
ing which a second spike can not occur, followed by a 
longer relative refractory period during which firing re- 
quires stronger perturbations. 




FIG. 3. First passage time distributions for the FHN (left 
panel) and I-F (right panel) models, forced with constant input. 
In both panels, (x) corresponds to high input and (o) to low. 

As previously argued, it is biologically reasonable to 
assume that noise affects the voltage-like variable. Thus, 
according to our argument, a higher steady input will 
result in a faster approach to the threshold crossing and 
therefore will reduce the probability of an untimely cross- 
ing. This is confirmed in Fig. 3, where the distribu- 
tions of periods are plotted for two inputs amplitudes. 
Note the large change in variance (1.5 to 2.34, 56%) for 
comparably small change in the average period (36.4 to 
39.1, 7%) in going from high to low input, also note that 
the low input distribution has a supra-exponential tail. 
For comparison, the distribution of inter-spike intervals 
corresponding to the I-F model with constant input is 
shown. In both cases, the distribution for high input 



tends to a Gaussian (parabola on semi-log plot), while the 
low input has an exponential tail (linear on semi-log plot) 
as expected for Poisson-like statistics. It is interesting to 
notice that even though this phenomenon will affect more 
strongly systems with zero-frequency quiescent-to-firing 
bifurcations, given the longer time to integrate fluctua- 
tions at low input (i.e. the system can evolve near a ho- 
moclinic orbit), the example presented here shows that it 
also affects non-zero-frequency transitions like the Hopf 
bifurcation present in the FHN model. 

III. EXPERIMENTAL RESULTS 

Thus, in neural data, one should expect to find that 
the output noise generically depends on the structure of 
the input. Experiments discussed next demonstrate the 
relevance of this observation for brain function. We re- 
call that an assumption usually made in the application 
of information theory to neurons is that the noise in- 
troduced by the communication channel is independent 
of the message being transmitted, and now explore to 
what extent this assumption is valid in real neural data. 
The result is presented in Fig. 4, where different vi- 
sual stimuli (i.e., the four messages) were presented to 
an anesthetized cat while electrophysiological recordings 
were made in the lateral geniculate nucleus (LGN) , which 
is the second stage of processing in the visual pathway 
(see Q for methods). 

The raster plot in Panel A shows the response to a 
moving bar with high contrast and high speed, and Panel 
B shows the responses to a low contrast bar moving at 
low speed, both results are shown in a window of 200 
msec centered at the peak of the ON response. The 
responses in Panel A clearly display a higher temporal 
precision than those for the condition in Panel B. Two 
further conditions were also presented, for high contrast 
and low speed, and for low contrast and high speed. In 
the inset of Fig. 4 we quantify the noise for the four 
messages as the "jitter" of the response onset, which we 
define as the average of the absolute value of the posi- 
tion with respect to the mean of the first spike in each 
trial. The messages include two conditions for the veloc- 
ity and two for the contrast. For each velocity condition, 
the high contrast is more reliable than the low one: 8.4 
msec vs. 22.1 msec, and 16.8 msec vs. 23.8 msec. Note 
however that the high contrast/high speed and low con- 
trast/low speed stimuli, although they have a very sim- 
ilar spike response (4.17 vs. 3.86) have the largest dif- 
ference in jitter (8.4 msec vs. 23.8 msec). Alternatively, 
we can measure the entropy difference between the cor- 
responding first spike probability distributions (1 msec 
bins), which varies from 2.0 bits for low contrast /high 
speed vs. high contrast/high speed, to 0.6 bits for high 
contrast/low speed vs. high contrast/high speed. Similar 
results showing contrast-dependent temporal precision in 
LGN have been reported previously in [13], where it is 
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also proposed that the variability is due to intrinsic noise. 
We also show that messages with indistinguishable mean 
rate responses can show large differences in their vari- 
ability, in agreement with the hypothesis of the threshold 
crossing derivative as the relevant parameter, which can 
be independent of the firing rate for particular inputs. 
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FIG. 4. Experimental example showing that different "mes- 
sages" elicits different noise (i.e., spike temporal jitter). Results 
gathered from recordings in cat's LGN where neuronal action po- 
tentials are recorded in response to the presentation of a moving bar 
with different contrast and speed. The inset in Panel A shows, for 
the conditions tested, the temporal jitter versus average number of 
spikes (collected during the window of 200 ms); (•) corresponds to 
low contrast/high speed, (+) to high contrast/high speed, (v) to 
low contrast/low speed, and (A) to high contrast/low speed. No- 
tice the data from moving bars of different contrast originating very 
different jitter. The raw data for two of the experimental condi- 
tions are presented in the raster plots of Panels A and B. Responses 
obtained with a moving bar of low contrast moving at low speed 
are plotted in Panel B. The (less variable) responses obtained with 
a moving bar of high contrast moving at high speed are depicted 
in Panel A. 



IV. NETWORK ARCHITECTURE AND 
RELIABILITY 

The observations discussed in the previous section are 
restricted to the behavior of individual neurons. How 
can they be extended to networks of spiking elements? 
In principle, we can assume that input-dependent noise 
will be present in early stages of processing, given the rel- 
ative simplicity of the neural architecture. The relevance 
of this phenomenon in higher order areas remains to be 
explored; nevertheless, we may hypothesize that unless 
dedicated architectures are implemented to eliminate it, 
significant input-dependent noise will be ubiquitous in 
neural processing. 



Indeed, a recurring question in brain theory and com- 
puter theory is whether there are reliable ways to com- 
pute which use unreliable components. We shall now 
show explicitly the existence of a network topology for 
which the overall time reliability can be made arbitrarily 
better than that found in its individual neurons. 

Assume a single input source I(t), which "fans out" to 
N noisy neurons Xf, these in turn "fan in", or input into 
a single (also noisy) neuron Y. Let's imagine an I(t) con- 
structed so that the X% fire once each at times £j. If the 
size of the connections X — ► Y is set so that Y fires after 
seeing half of the expected number of spikes coming from 
the Xs, then clearly neuron Y will be fire near the median 
time of the spike times £j. The median, as a descriptor of 
the ti, has a number of exceedingly nice properties in this 
context: (a) its variance decreases as 1/y/N, and thus the 
timing accuracy of Y can be made better than the indi- 
vidual Xs simply by choosing an appropriately large N; 
(b) the median, unlike the mean, is exceedingly robust 
against outliars and heavy-tailed noise; (c) the median 
does not require that it see the entire set ti but only half 
of it, and (d) the median is expected to lie at the time of 
highest concentration of the ti , thus maximizing the tim- 
ing accuracy of Y. This can be fully apreciated in Fig. 
5. Thus, we see that different topologies will propagate 
spike timing accuracy in different fashions, and that one 
probably should expect architectural correlates in place 
in the brain. 

To gain further insight into this issue and to study the 
implications of our observations on the synchronization 
of a neuronal network, we now concentrate on random 
but fixed time delays across pathways, rather than on in- 
dividual timing accuracy. We modify the previous model: 
all neurons of the middle layer are now noise-free and in 
principle identical, and so are their spike trains; the neu- 
rons are integrate-and-fire units as in Eq. 1, with the 
further addition of a refractory period. We next model a 
dispersion of the transmission delay times by rigidly shift- 
ing the spike train of each individual neuron by a ran- 
dom value uniformly distributed between and 5. The 
total charge that the integrating neuron receives does not 
depend on this shift, but when all neurons are synchro- 
nized, the derivative of the charge will be high and then 
we expect more reliable responses. Fig. 6 shows how the 
timing reliability (open circles), measured by the method 
of spike metrics 13 , decreases with the decorrelation of 
the input layer. Interestingly, when the derivatives are 
high the response is more reliable, but by the same token, 
the current supplied during the refractory period will be 
lost. This means that smooth distributions will result 
in an increased effective charge and thus the rate of re- 
sponse of the integrating neuron is bigger when the input 
layer is decorrelated, as shown in Fig. 6 (open squares) 
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FIG. 5. Simple model to study the effects of synchronization 
on the reliability of the neuronal responses. 75 Neurons synapse 
on a unique neuron modeled as a leaky integrate-and-fire as in Eq. 
1, with the addition of a refractory period; g=0.005/s, o-=0.07, re- 
fractory period = 10 ms. Mean Rate and Reliability for a neuron 
integrating spikes over an input layer of 75 neurons. A pattern of 
spikes was randomly generated (3.75 Hz) and each neuron of the 
input layer feeds the same pattern of spikes with a delay S. The 
value of the other parameters are as in Fig 5. When the input 
layer is synchronized, V of the integrating neuron is big and the 
response is very reliable. In this case, part of the charge from the 
input layer is lost because there is no summation during the re- 
fractory period. When 5 increases, the reliability decreases but the 
mean firing rate is increased. This suggests that a function of syn- 
chronization, which is known to occur in many different neuronal 
networks, might be to increase the reliability on the system at the 
expense of the intensity of the signal. 



1 There is a second regime (not shown), if the decorrelation 
is large compared to the decay time of the neuron, in which 
rate decreases with increasing 8 due to current loses. 
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FIG. 6. Reliability of a fan-out/fan-in structure. An input I(t) 
is fanned out in parallel to N identical neurons; one extra (identi- 
cal) neuron fans-in: it receives input from all N, with the weights 
adjusted so as to fire once per N/2 input spikes. In this example, 
N = 51 and I(t) is a constant plus an Ornstein-Uhlenbeck process. 
Top panel: rasters for one instance and all 51 neurons, to illustrate 
the reliabilty of the middle layer. Middle panel: raster for the out- 
put neuron, 40 instances. We emphasize that the output neuron 
has identical parameters to the 51 neurons in the middle layer, in- 
cluding identical levels of internal noise. Bottom panel: a noiseless 
counter firing twice per 51 spikes will implement a median mea- 
surement. Notice that both middle and bottom panels now show 
two spikes per each spike of the middle layer neurons, and one of 
them (the closest to the median) is the most reliable. 

We think that this very simple concept might be of 
physiological relevance, in that it implies a trade-off be- 
tween temporal reliability and the dynamic range of the 
signal: when the dynamic range is large, it is possible to 
afford the cost of losing part of the charge to achieve reli- 
ability, while in a situation in which the dynamic range is 
limited, reliability has to be sacrificed in order to preserve 
and propagate the signal. This suggests the possibility 
of integration pathways which multiplex the signal, as 
found in retinal adaptation. 



V. CONCLUSION 

The relevance of the message-dependent nature of 
noise in spiking elements is, to our understanding, two- 
fold. One aspect is its consequences for the use of Shan- 
non's Information Theory as a framework to measure the 
information content present in the output of a neuron. 
This is the subject of much work recently |15|. In this 
regard, a basic assumption commonly made is that the 
noise introduced by the communication channel is inde- 
pendent of the message being transmitted, which allows 
the modeling of a neuron as a Gaussian channel Jl6| , p7t . 
This is indeed desirable given that departures from the 
Gaussian channel have proved to be rather cumbersome; 
for instance, no general theory for the computation of 
channel capacity exists ]lc| . Our results render average 
measures of information per spike less meaningful than 
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usually thought, and speak for the necessity of concen- 
trating on particular individual spikes. This is specially 
critical when the message consists of only a few spikes 
(for instance, in the auditory pathway), or in the event 
of fast conductance modulations |l9| , which can dramat- 
ically affect the reliability. Thus, perhaps the most fun- 
damental consequence of Mainen & Sejnowski's result ||] 
is the implicit demonstration that cortical neurons are 
not always classical Gaussian channels. 

A second relevant aspect is the possible anatomical 
correlate of this phenomenon. In this regard, we have 
shown that unless dedicated architecture is implemented 
to reduce the multiplication of noise along the process- 
ing pathways, the encoding of behaviorally relevant (not 
only "artificial" stimuli) can be highly degraded. Finally, 
we demonstrated that it is possible to design network 
topologies with arbitrarily large temporal reliability, and 
consequently one should expect an evolutionary pressure 
to implement them in specific areas of the brain where 
time accuracy is essential. 
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